Software testing is essential to developing quality software products. Software testing involves operating a software application or a software system under a set of controlled conditions and evaluating the results. There are a variety of software tests which software applications may be subjected to, e.g., black box testing, white box testing, integration testing, regression testing, stress and load testing, usability testing, etc. Each software test allows a measurement of the actual performance of the software application versus desired performance of the software application.
Testing tools are another aspect of software testing. A variety of automated tools exist for testing software applications, e.g., code analyzers, coverage analyzers, memory analyzers, load and performance tools, web test tools, test case management tools, bug reporting, etc. Many of the tools mentioned above use static analysis, which involves inspection of source code of software when the software is not being executed.
Testing, and in particular, software testing in conjunction with testing tools and metrics, is often introduced in the latter stages of a software development process. There are often separate groups or teams involved in the software development process, e.g., a development team, a management team, and a testing team. The development team includes software engineers, programmers, etc. The management team may include software architects, team leaders, project managers, etc. The testing team includes test engineers, software testers, code developers, etc.
FIG. 1 shows a flow chart of a typical software development process. The process begins with the initial concept of a software system (Step 10). The necessary requirements of the software system are determined (Step 12), and functional design of the software system is prepared (Step 14). Additional designs are generated for internal functions, and plans for the documentation and tests are prepared (Step 16). The management team is typically involved in generating designs and documentation plans.
The software system is coded (Step 18). Typically, a development tool, such as an Integrated Development Environment (IDE) is used by members of the development team, especially code developers. The IDE is a programming environment that has been packaged as a software application, and often normally includes a code editor, a compiler, a debugger, and often a Graphical User Interface (GUI) builder. IDE's vary in complexity, from a simple editor with a compiler and possibly debugger extensions, to a complex development application which includes facilities to support teamwork, such as access to a shared code depository.
Next, documentation supporting the system is prepared (Step 20), and then the software testing begins (Step 22). The testing team or a group aside from the development team usually performs testing, but in some instances the development team may perform testing. Any maintenance or updates are implemented as a result of the testing (Step 24), i.e., identified bugs are addressed.
The software system may be re-tested (Step 26) and additional maintenance is implemented, if necessary (Step 28). The software development process described above is commonly known as a Systems Development Life Cycle model (SDLC model) and occasionally referred to as a “waterfall model.” The SDLC model is an iterative process and usually proceeds in strict order.
Often, use of an SDLC model for software development results in a flow of information such that the development team and the testing team communicate with each other. For example, the management team relays requirements and designs to the development team. Similarly, the management team relays testing requirements and test plans to the testing team. The test requirements and test plans include criteria by which the software application is to be evaluated. Once software testing begins, issues regarding performance found by the testing team are discussed with the management team and relayed back to the development team, as needed.
Testing tools are often used to determine values of metrics. Metrics are often used to provide objective criteria for evaluation of software applications. Metrics focus on issues of complexity, extensibility, modularity, readability, etc. Particular aspects of software applications may be quantified using metrics attributes. Metrics attributes are measurements used in software tests to locate potential problem areas within the software application. Different than a “bug,” metrics attributes are desired standards in the design and operation of the software application, rather than mistakes or errors in the functionality of the software application. Testing “good code” is not purely a consideration of error free code, instead “good code” is considered for complexity, extensibility, modularity, readability, etc. Testing tools are often used to determine values of metrics attributes. Formulas may be used to calculate numeric values for particular metrics attributes.
Metrics may be oriented towards a particular design or programming paradigm. For example, object-oriented metrics attributes are metrics designed specifically to evaluate the performance of object-oriented code. Examples of objected-oriented metrics attributes include Depth of Inheritance Tree (DIT), Number of Children (NOC), Coupling Between Objects (CBO), Weighted Methods per Class (WMC), Response for Class (RFC), Message Passing Coupling (MPC), etc. Each of the object-oriented metrics attribute provides different information as to quality of the object-oriented code. The information may relate to functions, classes, or an entire software system. For example, MPC is a measure of message passing among classes, indicating a measure of coupling between a class and an entire software system. Determining a value of MPC provides an indication of software application complexity. Although a measured value of a metrics attribute may be out-of-bounds (either high or low), the software application may not necessarily be flawed. Metrics attributes give guidance as to desired standards, but are not necessarily mandates.
Many techniques may be used to determine values of metrics attributes. One such technique is static analysis. Static analysis is typically used to locate problem areas in the source code or compiled code, e.g., potential bugs, unnecessary complexity, or high maintenance areas. Several mechanisms (similar to compilers) are used to perform static analysis, e.g., lexical or binary scanning, parsing, semantic analysis, data flow analysis, etc.
For example, after testing an object-oriented software application, the value of the MPC metrics attribute between Class X and Class Y may be found to be high, indicating Class X and Class Y are strongly coupled. Strongly coupled classes are closely tied together, so that modifications attempted to one class are more likely to affect the other class, should modification be required. In a typical scenario, the MPC metrics attribute, along with other metrics attributes, are compiled into performance reports and discussed with the management team. The management team suggests improvements to problem areas in the source code, e.g., to improve the message coupling between Class X and Class Y. The improvements are communicated to the development team for resolution.
Metrics reports are often used in software development to improve software quality. The metrics reports may be generated at certain points during the development in cycle (e.g., approximately halfway between start of development and the expected release date of the software). Such metrics reports may be presented as feedback to the management team. There is a definite cost associated with time, money, resources (tools and people), and space expended to develop a software system using the aforementioned process.